Physical configuration of a device for interaction mode selection

ABSTRACT

In various embodiments, methods and systems are provide for detecting a physical configuration of a device based on sensor data from one or more configuration sensors. The physical configuration includes a position of a first display region of the device with respect to a second display region of the device, where the position is physically adjustable. A configuration profile is selected from a plurality of configuration profiles based on the detected physical configuration of the device. Each configuration profile is a representation of at least one respective physical configuration of the device. An interaction mode corresponding to the selected configuration profile is activated, where the interaction mode includes a set of mode input/output (I/O) features available while the interaction mode is active. Device interfaces of the device are managed using at least some mode I/O features in the set of mode I/O features based on the activating of the interaction mode.

BACKGROUND

Computing devices are increasingly being equipped with automatedpersonal assistant software. For example, APPLE® has developed apersonal assistant referred to as “Siri,” GOGGLE® has developed apersonal assistant referred to as “Voice Actions” or “Google Now,”MICROSOFT® has developed a personal assistant referred to as “Cortana,”and AMAZON® has developed a personal assistant referred to as “Alexa.”Typically personal assistants are optimized to interact with users onspecific types of devices and in specific use cases. For example, apersonal assistant accessed through a phone may be optimized differentlythan a personal assistant accessed through an environmental device, suchas a living room device.

When a personal assistant, or other type of user agent, is used with adevice in a way that it is not optimized for, the user experience cansuffer, along with the performance of the user agent, and the deviceitself. For example, if a phone is placed away from a user in theenvironment, the personal assistant may be unable to interpret theuser's speech when optimized for near-field speech recognition.Furthermore, the personal assistant may be unable to use features thattake advantage of the device's place in the environment in order tobetter assist and interact with the user. Additionally, some activefeatures could be unimportant or unwanted in certain user and/orenvironmental contexts, thereby wasting computing resources, such aspower and computation.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Aspects of the present disclosure relate to utilizing the physicalconfiguration of a device to select an interaction mode for the device,such as for interaction with a user agent using the device. In doing so,features can be made available that are optimal for the particularphysical configuration of the device.

In some implementations, interaction modes are selected based on thephysical configuration of a device. An interaction mode includes a setof features available to manage device interfaces, such as displays,microphones, speakers, and other components while in the interactionmode. Different physical configurations of the device correspond todifferent interaction modes that have different sets of features. Whenthe physical configuration of a device is detected, an appropriateinteraction mode is activated for that configuration.

In some respects, the physical configuration of a device can include anycombination of a physical relationship between the device and itsenvironment (also referred to as a physical environmentalconfiguration), and a physical relationship between portions of thedevice (also referred to as physical device configuration). An exampleof a physical environmental configuration is an orientation of a devicewith respect to its environment. An example of a physical deviceconfiguration is a position of one portion of a device with respect toanother portion of the device.

In some respects, a device includes multiple display regions, such asmultiple displays, where the position of the multiple display regionswith respect to one another is adjustable. As one example, two displayscan be connected by a hinge that allows an angle between the displays tobe adjusted. As another example, a single display can be bendable toallow an angle between display regions to be adjusted. A new physicalconfiguration of a device may be reached by adjusting the angle, and asa result, an active interaction mode for the device may be switched toan interaction mode corresponding to the new physical configuration.These and other concepts are contemplated as being within the scope ofthe present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is described in detail below with reference to theattached drawing figures, wherein:

FIG. 1 shows a block diagram showing an example of an operatingenvironment in accordance with embodiments of the present disclosure;

FIG. 2A shows a device in a physical configuration in accordance withembodiments of the present disclosure;

FIG. 2B shows a device in a physical configuration in accordance withembodiments of the present disclosure;

FIG. 3A shows a device in a physical configuration in accordance withembodiments of the present disclosure;

FIG. 3B shows a device in a physical configuration in accordance withembodiments of the present disclosure;

FIG. 4 is a flow diagram showing a method in accordance with embodimentsof the present disclosure;

FIG. 5 is a flow diagram showing a method in accordance with embodimentsof the present disclosure; and

FIG. 6 is a block diagram of an exemplary computing environment suitablefor use in implementations of the present disclosure.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

Aspects of the present disclosure relate to utilizing the physicalconfiguration of a device to select an interaction mode for the device,such as for user interaction with a user agent accessible through thedevice. In doing so, features can be made available that are optimal forthe particular physical configuration of the device.

In some implementations, interaction modes are selected based on thephysical configuration of a device. An interaction mode includes a setof features available to manage device interfaces, such as displays,microphones, speakers, and other components while in the interactionmode. Different physical configurations of the device correspond todifferent interaction modes that have different sets of features. Whenthe physical configuration of a device is detected, an appropriateinteraction mode is activated for that configuration.

In some respects, the physical configuration of a device can include anycombination of a physical relationship between the device and itsenvironment (also referred to as a physical environmentalconfiguration), and a physical relationship between portions of thedevice (also referred to as physical device configuration). An exampleof a physical environmental configuration is an orientation of a devicewith respect to its environment. An example of a physical deviceconfiguration is a position of one portion of a device with respect toanother portion of the device.

In some respects, a device includes multiple display regions, such asmultiple displays, where the position of the multiple display regionswith respect to one another is adjustable. As one example, two displayscan be connected by a hinge that allows an angle between the displays tobe adjusted. As another example, a single display can be bendable toallow an angle between display regions to be adjusted. A new physicalconfiguration of a device may be reached by adjusting the angle, and asa result, an active interaction mode for the device may be switched toan interaction mode corresponding to the new physical configuration.

Turning now to FIG. 1, a block diagram is provided showing an example ofan operating environment in which some implementations of the presentdisclosure may be employed. It should be understood that this and otherarrangements described herein are set forth only as examples. Otherarrangements and elements (e.g., machines, interfaces, functions,orders, and groupings of functions, etc.) can be used in addition to orinstead of those shown, and some elements may be omitted altogether forthe sake of clarity. Further, many of the elements described herein arefunctional entities that may be implemented as discrete or distributedcomponents or in conjunction with other components, and in any suitablecombination and location. Various functions described herein as beingperformed by one or more entities may be carried out by hardware,firmware, and/or software. For instance, some functions may be carriedout by a processor executing instructions stored in memory.

Among other components not shown, operating environment 100 includesuser device 102, network 104, interaction mode manager 106, and server108. Interaction mode manager 106 includes configuration detector 120,interaction mode selector 122, configuration profiles 124, interfacemanager 128, and configuration sensors 130.

It is noted that although depicted separately, in some embodiments, allof the functionality of operating environment 100 is integrated intouser device 102. Thus, for example, storage 180 and interaction modemanager 106 may be in user device 102. Server 108 and network 104 maynot be required for these embodiments. However, in other cases, some ofthe functionality of operating environment 100 is external to userdevice 102, such as some of the functionality of interaction modemanager 106.

Thus, it should be understood that operating environment 100 shown inFIG. 1 is an example of one suitable operating environment. Each of thecomponents shown in FIG. 1 may be implemented via any type of computingdevice, such as one or more of computing device 600, described inconnection to FIG. 6, for example. These components may communicate witheach other via network 104, which may be wired, wireless, or both.Network 104 can include multiple networks, or a network of networks, butis shown in simple form so as not to obscure aspects of the presentdisclosure. By way of example, network 104 can include one or more widearea networks (WANs), one or more local area networks (LANs), one ormore public networks, such as the Internet, and/or one or more privatenetworks. Where network 104 includes a wireless telecommunicationsnetwork, components such as a base station, a communications tower, oreven access points (as well as other components) may provide wirelessconnectivity. Networking environments are commonplace in offices,enterprise-wide computer networks, intranets, and the Internet.Accordingly, network 104 is not described in significant detail.

It should be understood that any number of user devices, servers, anddata stores may be employed within operating environment 100 within thescope of the present disclosure. Each may comprise a single device ormultiple devices cooperating in a distributed environment. For instance,one or more of the constituents of interaction mode manager 106 may beprovided via multiple devices arranged in a distributed environment thatcollectively provide the functionality described herein. Additionally,other components not shown may also be included within the distributedenvironment.

In some embodiments, one to all of the functionality of interaction modemanager 106 can be integrated, at least partially, into a user device,such as user device 102. In some embodiments, operating environment 100does not require network 104. For example, storage 180 and interactionmode manager 106 could be completely integrated into user device 102. Inother cases, interaction mode manager 106 may at least partially be acloud computing service (e.g., integrated into server 108).

Storage 180 can store any data utilized by interaction mode manager 106and/or software 110 and may be at least partially located on user device102 and/or server 108. The data can include sensor readings,configuration profiles, mode I/O features, settings, deviceconfigurations, and the like, any of which may be persistently stored onstorage 180 for use by the various components of operating environment100.

User device 102 can be a client device on a client-side of operatingenvironment 100, while server 108 can be on a server-side of operatingenvironment 100. When implemented using server 108, interaction modemanager 106 can comprise server-side software designed to work inconjunction with client-side software on user device 102 so as toimplement any combination of the features and functionalities discussedin the present disclosure. An example of such client-side software issoftware 110 on user device 102. As used herein, software 110 broadlyincludes one or more of any combination of an operating system, a systemservice, an application (e.g., installed on an operating system of thedevice), firmware, a device driver, and the like. When implementedwithout using server 108, similar functionality as server 108 may beachieved locally by software 110.

In implementations where a digital agent is employed, software 110 cancorrespond to the digital agent. Further the digital agent can beimplemented using any combination of client-side and/or server-sidesoftware. Where passive listening is employed by the digital agent, insome cases the client-side software performs passive listening byprocessing audio from one or more users. In some cases, the client-sidesoftware further determines whether to enter into an active listeningmode based on the processing of the audio, such as by identifying a“wake up” command from the audio. In the active listening mode, theclient-side software may optionally provide audio received during theactive listening mode to server-side software for processing. Furtherthe server-side software may provide one or more instructions to theclient-side software in response to the processing.

User device 102 comprises any type of computing device capable of beingoperated by a user. In some implementations, user device 102 is the typeof computing device described in relation to FIG. 6 herein. By way ofexample and not limitation, a user device may be embodied as a personalcomputer (PC), a laptop computer, a mobile device, a smartphone, atablet computer, a smart watch, a wearable computer, a personal digitalassistant (PDA), an MP3 player, a global positioning system (GPS) ordevice, a video player, a handheld communications device, a gamingdevice or system, an entertainment system, a vehicle computer system, anembedded system controller, a remote control, an appliance, a consumerelectronic device, a workstation, any combination of these delineateddevices, or any other suitable device.

The user device includes one or more processors, and one or morecomputer-readable media. The computer-readable media includescomputer-readable instructions executable by the one or more processors.The instructions may be embodied by one or more software modules, suchas software 110 shown in FIG. 1. Software 110 is referred to in singularform for simplicity, but its functionality can be embodied by one ormore software modules in practice. As indicated above, in some cases,the software is embodied collectively by user device 102 and server 108(and/or optionally other devices).

Interaction mode manager 106 includes configuration detector 120, whichis configured to detect whether a device, such as user device 102 is ina physical configuration and/or determine a current physicalconfiguration of the device. As used herein, a physical configuration ofa device includes at least one physical relationship that must besatisfied for the device to be in the physical configuration. A physicalconfiguration of a device can include any combination of a physicalrelationship between the device and its environment (also referred to asa physical environmental configuration), and a physical relationshipbetween portions of the device (also referred to as physical deviceconfiguration).

FIG. 2A shows an example of device 202 in one physical configuration andFIG. 2B shows an example of device 202 in another physicalconfiguration. Similarity, FIG. 3A shows an example of device 302 in onephysical configuration and FIG. 3B shows an example of device 302 inanother physical configuration. Each of the physical configurationsshown can be detected by interaction mode manager 106. Further device202 and device 302 are different examples of user device 102 of FIG. 1.

The physical configurations shown in FIGS. 2A, 2B, 3A, and 3B eachcomprise a physical device configuration. In the examples shown, eachphysical configuration corresponds to a physical relationship betweendisplay regions of the corresponding device, and in particular, aposition of a first display region of the device with respect to asecond display region of the device. In FIGS. 2A and 2B, the displayregions correspond to respective displays of device 202. For example,one display region may correspond to display 242A and another displayregion may correspond to display 242B. The physical relationship betweenthose display regions can correspond to angle 250 between the displaysin FIG. 2A and angle 252 between the displays in FIG. 2B.

In contrast to FIGS. 2A and 2B, FIGS. 3A and 3B show a physicalrelationship between display regions located on a common display of adevice. For example, the physical configuration of FIG. 3A correspondsto angle 350 between display regions 342A and 342B, which bothcorrespond to display 342. Similarly, the physical configuration of FIG.3B corresponds to angle 352 between display regions 342A and 342B, whichboth correspond to display 342.

Each of devices 202 and 302 are designed to be physically manipulated,such as by a person, in order to adjust the physical configuration ofthe device. Thus, the device may be manipulated into and out of thephysical configurations shown. In the present examples, the physicalmanipulation comprises bending the device. Device 202, for example, is abendable device comprising hinge 244, which can be manipulated to adjustthe position (e.g., angle and/or distance) between displays 242A and242B. Any suitable type of hinge or joint can be employed such as adiscrete hinge or one integrated into a display or other portion ofdevice 202.

In the example shown, hinge 244 is fully articulating, allowing device202 to be adjusted from fully closed position, in which displays 242Aand 242B are facing one another and unviewable to a user, to a fullyopen position, in which displays 242A and 242B are opposing one another,and exposed for viewing. Each achievable configuration of the devicecorresponds to a respective physical configuration of the device.

Similar to device 202, device 302 is also a bendable device. Device 302comprises bendable display 342, which can be manipulated to adjust theposition (e.g., angle and/or distance) between display regions 342A and342B. Any suitable bendable display can be employed. Further, in somecases, display 342 can be bent in different manners than what is shown,including about other axes. The physical configuration of FIG. 3A mayvary from FIG. 3B in that device 302 is bent in an inverted manner fromFIG. 3A to FIG. 3B.

It should be appreciated that in various implementations, the form ofdevices as well as the physical device configurations available for thedevices can vary widely while still being within the scope of thepresent disclosure. Further, the means for achieving the physical deviceconfigurations can also vary widely from the examples of a hinge andbendable display shown. As an additional example, a composite or unitarydevice could comprise two or more segments, which can be detached,separated, repositioned, and/or reattached by the user in order toachieve corresponding physical device configurations.

The physical configurations of devices of FIGS. 2A, 2B, 3A, and 3B alsoinclude physical environmental configurations. In FIGS. 2A and 2B,angles 250 and 252 may be substantially similar, as an example, so thatdevice 202 is in the same physical device configuration in both FIGS. 2Aand 2B; however, they may correspond to different physicalconfigurations of device 202 because they have different physicalenvironmental configurations. In particular, the orientation of device202 with respect to its environment varies between FIGS. 2A and 2B. InFIG. 2A, the physical configuration of device 202 may be referred to asa “brochure” posture and the physical configuration of device 202 inFIG. 2B may be referred to as an “easel” or “tent” posture.

In the physical configuration of FIG. 2A, an edge of device 202 issituated on support surface 260. The edge is bisected by an axis aboutwhich device 202 can be bent (e.g., corresponding to hinge 244).Further, in this example, device 202 comprises two minor edges and twomajor edges, where a minor edge is in contact with support surface 260.In the present example, angle 250 allows device 202 to be fullysupported on support surface 260 by the panels comprising displays 242Aand 242B at the edge to remain in the configuration as shown.

In the physical configuration of FIG. 2B, two edges of device 202 aresituated on support surface 260. In this example, device 202 comprisesthe two minor edges of device 202 are in contact with support surface260. Device 202 is fully supported on support surface 260 on the twoedges to remain in the configuration as shown. Such support may beprovided, for example, at least partially by hinge 244. For example,hinge 244 may have sufficient friction that prevents device 202 fromchanging its physical device configuration (e.g., prevents angle 252from changing).

In both FIGS. 3A and 3B, device 302 has the same orientation withrespect to support surface 360. In other cases, device 302 could besupported by two edges, similar to device 202 in FIG. 2B, supported byat least a minor edge, or take on another potential physical posture orconfiguration.

In should be appreciated that the physical environmental configurationsshown are merely exemplary. Further, in various implementations, adevice may include any suitable support structure, such as one or morestands, to support a physical environmental configuration. Further, itis contemplated that a physical environmental configuration does notrequire a support surface and could generally correspond to a positionof a device in three dimensional space.

It is further noted that the devices shown are unitary devices. However,in other cases, device 202, for example, could be a composite device.For example, display 242A could be on one device and display 242B couldbe on another device. The devices may be coupled together and in somecases completely severable. For example, two separate devices may beconnectable to achieve a composite device. For the purposes of thepresent disclosure, the term “device” refers to a composite device or aunitary device (i.e., a non-composite device).

As an overview, configuration detector 120 can detect a physicalconfiguration of a device based on sensor data from configurationsensors 130. Configuration detector 120 selects a configuration profilerepresenting the detected physical configuration, where theconfiguration profile defines an interaction mode for the device.Interaction mode selector 122 activates the interaction modecorresponding to the detected configuration profile. The interactionmode represents one or more mode Input/Output (I/O) features of mode I/Ofeatures 126 of the device. Activating the interaction mode ensures thecorresponding mode I/O features are available to interface manager 128,which manages device interfaces of the device using available mode I/Ofeatures. Activating the interaction mode may activate one or more modeI/O features of the interaction mode and/or deactivate one or more modeI/O features based on those features not being included in theinteraction mode. Activating a mode I/O feature can include activating acomputing process and/or hardware corresponding to the mode I/O feature.For example, available features can be switched from those included inan interaction mode corresponding to a previous physical configurationof the device to those included in the interaction mode for the currentphysical configuration of the device. Thus, the device can be physicallymanipulated to switch between or set interaction modes.

Configuration detector 120 can detect physical configurations usingsensor data from at least one configuration sensor, such as anycombination of configuration sensors 130. Each of configuration sensors130 correspond to a physical sensor component of a device, such as userdevice 102. Configuration detector 120 can determine whether user device102 is in a physical configuration based on at least one signal from thephysical sensor component. Examples of configuration sensors thatconfiguration detector 120 can use to detect a physical configuration ofuser device 102 include one or more of any combination of a camera, atouch sensor, a microphone, a motion sensor, a light sensor, a forcesensor, a gyroscope, an Internal Measurement Unit (IMU), a GlobalPositioning System (GPS) receiver, a pressure sensors, and many more.

As one specific example, configuration detector 120 may detect thatdevice 202 is in the physical configuration shown using one or morepressure sensors on the edge of device 202 in contact with supportsurface 260 in FIG. 2A to detect the physical environmentalconfiguration and an IMU integrated into hinge 244 to detect thephysical device configuration. The physical device configuration of FIG.2B could similarly be detected and the physical environmentalconfiguration could be detected using one or more pressure sensors on atleast one of the edges of device 202 in contact with support surface 260in FIG. 2B.

It should be appreciated that many approaches exist for detecting thephysical device configuration and/or physical environmentalconfiguration of a device. As another example, one or more of cameras264A and 264B and/or gyroscopes of device 202 could be used to detectthe physical configuration of device 202. Further, device 202 may lockinto one or more discrete positions, which when achieved provides asignal to configuration sensors 130 to indicate the physicalconfiguration. In one example, the minor edges of device 302 could snapor lock together indicating a physical configuration in which device 302(e.g., display 342) forms a continuous loop. In some cases,configuration detector 120 explicitly detects a physical configurationfrom one or more configuration sensors. However, configuration detector120 may infer a physical configuration from any combination of sensordata along with other information available to the device. As oneexample, the physical configuration of FIG. 2A could be inferred, atleast partially based on configuration detector 120 determining device202 has not moved for a predetermined amount of time (e.g., using motionsensors, gyros, and/or cameras). As another example, the physicalconfiguration of FIG. 2A could be inferred, at least partially based onconfiguration detector 120 determining device 202 has not been touchedfor a predetermined amount of time (e.g., using touch sensors).

Each physical configuration detected by interaction mode manager 106 cancorrespond to one of configuration profiles 124. Configuration profiles124 include configuration profiles, examples of which includeconfiguration profiles 124A and 124B. Each configuration profilerepresents one or more physical configurations of a device. For exampleconfiguration profile 124A represents at least the physicalconfiguration of device 202 in FIG. 2A and configuration profile 124Arepresents at least the physical configuration of device 202 in FIG. 2B.It is noted that in the present example, a single physical configurationis represented by a single configuration profile. However, a singleconfiguration profile can represent multiple physical configurations.For example, configuration profile 124A could represent device 202through a range of values for angle 250. Configuration profile 124Acould also exclude one or more ranges of values for angle 250,particularly in cases where configuration detector 120 can utilizemeasurements of angle 250 from one or more configuration sensors.

Each example of a physical configuration described herein could berepresented by a respective configuration profile or combined into oneor more configuration profiles with another physical configuration.Thus, the physical configurations in both FIGS. 3A and 3B couldcorrespond to the same configuration profile, or could correspond toseparate configuration profiles. In various embodiments, configurationdetector 120 is capable of detecting physical configurationssufficiently (e.g., with sufficiently granularity) to select or identifyat most a single configuration profile from configuration profiles 124Athat represents the current physical configuration of the device. Indoing so, configuration detector 120 may be capable of distinguishingbetween each physical configuration represented by a configurationprofile.

Each configuration profile defines an interaction mode for the one ormore physical configurations represented by the profile. An interactionmode represents a set of mode I/O features, which includes at least onemode I/O feature. In the present example, the set of mode I/O featuresare a subset of mode I/O features 126. Each configuration profile cancorrespond to a different set of mode I/O features. For example,configuration profile 124A can comprise mode I/O features 126A, 126B,and 126C and configuration profile 124B can comprise mode I/O features126B and 126D. Any number of mode I/O features may be included in a setof mode I/O features for an interaction mode.

The set of mode I/O features for an interaction mode of a configurationprofile represents a set of mode I/O features available to interfacemanager 128 while the device is in a physical configuration of aconfiguration profile detected by configuration detector 120. A mode I/Ofeature represents one or more functions that use data from or providesdata to at least one input or output component of the device. Usingconfiguration profiles, the mode I/O features available in physicalconfigurations can be set such that they are optimal for the physicaland/or environmental characteristics of the device as well as expecteduse cases.

In some implementations, each configuration profile is predefined forphysical configurations with corresponding mode I/O features. However,at least some configuration profiles and/or mode I/O features forassociated interaction modes can be generated at any time. For example,mode I/O features could be generated using associated rules that operatebased on sensor data from configuration sensors 130 and/or othercontextual data available to configuration detector 120. For example,configuration detector 120 could generate configuration profiles basedon device information that defines device I/O features the device iscapable of including hardware profiles, installed drivers, and the like.In some cases, configuration detector 120 selects one or morepre-generated configuration profiles to be available forselection/detection based on any combination of the aforementionedinformation. For example, some configuration profiles may be suitablefor some devices and not others. As another example a peripheral couldbe attached to a device or the device could be upgraded with hardware orsoftware and the configuration profiles could be updated and/or newconfiguration profiles could be added to available configurationprofiles. It should also be appreciated that in some embodiments, modeI/O features for pre-generated configuration profiles could be added toor removed from the configuration profiles based on any combination ofthe aforementioned information.

Having activated an interaction mode, interface manager 128 can managedevice interfaces of the device using at least some features in the setof features based on the activating of the interaction mode. Mode I/Ofeatures may be made available at any suitable level of abstraction. Oneor more of the mode I/O features may be part of the operating systemand/or firmware of the device (e.g., accessible by apps using an API).As another example, one or more of the mode I/O features may be part ofone or more applications, services, drivers, or other software that arenot part of the operating system. An available mode I/O feature may beinvoked by one or more designated applications or services, or madeavailable globally. In some cases, a mode I/O feature becomes availableby activating the mode I/O feature, which may include launching, waking,and/or enabling a service that performs one or more functions of themode I/O feature.

Although a set of features of an interaction mode may be available, notall of the mode I/O features may be employed. For example, mode I/Ofeatures may be used as needed, or as invoked, during an interactionsession with one or more users. However, one or more conditions forinvoking a mode I/O feature may not occur during a particularinteraction session (e.g., based on device inputs such as from the userand/or software that facilitates the interaction session).

In some cases, in response to detecting a physical configuration andselecting a corresponding configuration profile, the interaction mode isautomatically activated for the device. In other cases, the user ispresented with a prompt, such as an option to activate the interactionmode. A prompt could be used, for example, based on a level ofconfidence configuration detector 120 calculates on the device in aphysical configuration corresponding to the configuration profile. Wherethe level of confidence is exceeded, the interaction mode couldautomatically be activated. This approach may be especially suitable forinference based determinations of physical configurations.

In some implementations, activating an interaction mode automaticallydeactivates one or more mode I/O features. For example, a configurationprofile could include a list of mode I/O features to exclude fromavailability while its corresponding interaction mode is active (e.g.,while the device is in the corresponding physical configuration). Inaddition, or instead, activating an interaction mode may automaticallydeactivate a currently active interaction mode (e.g., of a configurationprofile). This may include deactivating any of mode I/O features 126that are not included in the newly activated interaction mode. Thus,interaction mode selector 122 can actively switch between interactionmodes as the physical configuration of the device changes andconfiguration detector 120 detects those changes (e.g., in real-time ornear real-time). It is noted in some cases, an interaction mode couldbecome active when no interaction mode was previously active. As oneexample, manipulating a device into a defined physical configurationcould activate an interaction mode.

Activating a mode I/O feature can correspond to activating a physicalcomponent of the device (e.g., switching a component on or waking itfrom sleep or hibernation, which increases power draw for thecomponent), such as one or more cameras, speakers, microphones,displays, or any combination thereof on the device, or activatingfunctionality corresponding to any of those devices. Similarly,deactivating a mode I/O feature can correspond to deactivating aphysical component of the device, such as one or more cameras, speakers,microphones, displays, or any combination thereof on the device, ordeactivating functionality corresponding to any of those devices.

An example of a mode I/O feature includes ambient or passive listening,such as for a user agent, which could be a user assistant like apersonal or group assistant. While ambient or passive listening isactive, interface manager 128 utilizes one or more microphones of userdevice 102 to, for example, detect a verbal prompt or other audible cueor speech input from a user, such as an activation word, phrase, orother activation prompt. Using implementations of the presentdisclosure, while device 202 is in either of the positions shown, aconfiguration profile could allow for ambient or passive listening to beactive. However, in a different physical configuration of device 202,ambient or passive listening may be inactive (or deactivated based onthe physical configuration). In one example, the physical configurationcould be when displays 242A and 242B are facing one another, such aswhen the device is in a fully closed position. In addition, or instead,a suitable physical configuration could be when displays 242A and 242Bare opposing one another, such as when the device is in a fully openedposition.

By activating and/or deactivating mode I/O features based on physicalconfigurations of the device, computing resources can be preserved suchas by deactivating mode I/O features that are unneeded or unimportantfor expected use cases of the device in certain configurations. Forexample, passive or ambient listening can be power intensive andtherefore disabling such functionality preserves power.

As another example, one or more mode I/O features can correspond to aprivacy level or setting. The privacy level or setting could dictatewhat content is or is not displayed on one or more of displays 242A and242B, or otherwise output via devices interfaces (e.g., audible contentover a speaker). When a privacy level of device 202 is increased by aphysical configuration of the device, for example, text messages,instant messages, emails and/or other user notifications may not bedisplayed on display and/or audibly indicated on device 202. This may besuitable for the physical configurations shown in FIGS. 2A and 2B, wherethe device may be part of a group meeting or expected use case thatinvolves at least one user in addition to or instead of a primary userof the device (e.g. a primary user could be a user currently logged intothe operating system, an owner of the device, a user associated with apersonal mode of the device, and/or a user identified by a specific userprofile).

One or more mode I/O features can also correspond to a camera operationsetting for the device. As one example, device 202 may switch from dualor multi camera setting to independent camera setting based on aphysical configuration of the device. For example, in the physicalconfiguration of FIG. 2A, the device may be in multi camera setting,where interface manager 128 operates cameras 264A and 264B inconjunction to interpret environmental data. This could include 3-Dreconstructions of the environment, such as to track locations of usersand/or objects in the environment. In the physical configuration of FIG.2B, the device may instead be in independent camera setting, whereinterface manager 128 operates at least one of cameras 264A and 264Bindependently to interpret environmental data. This could include 2-Dtracking or other environmental capture and/or interpretation. As anexample, in the physical configuration of FIG. 2B it may be expectedthat each camera is able to capture substantially independent views ofthe environment. Thus, a user visible to camera 264A might not bevisible to camera 264B. In contrast, in the physical configuration ofFIG. 2A, the view of each camera may substantially overlap, particularlywhere the device is in the corner of a room or another location withhigh visibility of the environment.

One or more mode I/O features can also correspond to a display operationsetting for the device. Similar to the example above, device 202 mayswitch from a dual or multi display setting to an independent displaysetting based on a physical configuration of the device. In a multidisplay setting, interface manager 128 operates at least two displays ofthe device as a single display. For example, displays 242A and 242B maybe operated as a single display in FIG. 2A under the assumption that auser can typically view both displays concurrently. In an independentdisplay setting, interface manager 128 operates multiple displays and/ordisplay regions of the same display as separate displays. For example,in FIG. 2B, displays 242A and 242B may be operated independently underthe assumption that a user can typically view only one of the displaysat a time. For example, each display or display region may be facing aparticular user and the content displayed thereon may be based on theparticular user being able to view the corresponding display. Further,the content might move from one display or region to another as the usermoves around the environment. For example, device 202 could track a useraround the environment comprising the device and modify the display(s)or region(s) as the user moves around (e.g., determining a positionand/or direction of the user and updating the display accordingly). Thiscan be done to ensure the content remains visible to the user.

In one implementation, the independent display setting is used for thephysical configuration of FIG. 2B to translate user speech. A first userfacing camera 264B may provide speech input to device 202. Optionally,the speech may be converted into text in the user's language anddisplayed on display 242B. The speech can also be automaticallytranslated into another language including converting the speech intotext of the other language and displayed on display 242A for viewing byanother user.

In some examples, each display displays at least some of the samecontent. For example, because each user may be unable to fully view aparticular region of the display, the content may be redundantlydisplayed so it is viewable from more locations in the environment.

Other examples of display operation settings include content displaysize settings. For example, the same content may be displayed larger insome physical configurations than in others on the device. As anexample, it may be presumed that when device 202 is fully opened it willbe used up close by a single user and therefore font, graphics, and thelike may be smaller than in the physical configurations of FIGS. 2A and2B. In addition to content display size, content display location can beset by a mode I/O feature including a display region on a display and/orwhich display to display the content.

Other examples of display operation settings include content displayorientation settings. For example, in the configuration of FIG. 2A aportrait orientation could be used and in the configuration of FIG. 2B alandscape configuration could be used for the entire display(s), regionsthereof, and/or particular items of content thereof.

One or more mode I/O features can also correspond to a microphoneoperation setting for the device. Similar to the example above, device202 may switch from dual or multi microphone setting to an independentmicrophone setting based on a physical configuration of the device. In amulti microphone setting, interface manager 128 operates one or moremicrophones of the device collaboratively. For example, an array ofmicrophones could be along an edge of device 202 (e.g., the top edgeproximate to cameras 264A and 264B and possibly on each panel of thedevice). In an independent display setting, interface manager 128operates the microphones independently. For example, in a multimicrophone setting, interface manager 128 may use multiple microphonesin conjunction to reduce interference between users who are speaking orotherwise better pinpoint locations of sounds in the environment.

In some respects, based on the physical configuration of the device,interface manager 128 may switch between a near field interaction modeand far field interaction mode. The near field interaction mode isoptimized for interacting with users proximate to the device, whereasthe fair field mode is optimized for interacting with usersnon-proximate to the device. A near field interaction mode couldcomprise a smaller content display size setting, a lower volume setting,enabled or optimized touch input control, a near field speech model, anindependent microphone mode, and the like, in contrast to a far fieldinteraction mode.

In some respects, based on the physical configuration of the device,interface manager 128 may switch between user interaction settings ofthe device. This can include switching from group interaction setting topersonal interaction setting, or between different types of groupinteraction settings. As used herein, a user interaction setting isdefined by at least one user. In a personal interaction setting, theuser is typically the primary user of the device and/or the usercurrently logged into the device. A group interaction setting is definedby multiple users. Group interaction settings may optionally becustomized for interaction with one or more particular users. In somecases, user interaction settings are based on determining environmentalcharacteristics of the device, such as based on optionally detecting atleast one to all of the users in the environment (e.g., usingenvironmental data captured from the environment by the device) orotherwise inferring their presence. Thus, the appropriate settings maybe applied in the interaction mode based on detecting correspondingenvironmental characteristics (which could change and be updatedaccordingly).

In some cases, one group interaction setting is a family interactionsetting and includes mode I/O features optimized for a family. Anothergroup interaction setting may be a meeting interaction setting andinclude mode I/O features optimized for colleagues, clients, businessassociates, and the like. It is noted that individual users for userinteraction settings may be predefined and have corresponding userprofiles and/or settings for one or more interaction settings that couldbe enforced when the user is present. However, user interaction settingsneed not be explicitly defined and the presence of particular users orsystem defined users not be required.

In some cases, switching from a personal interaction setting to a groupinteraction setting increases the privacy level and/or activates privacysettings for device interfaces of the device. In particular each userinteraction setting could have corresponding privacy settings or levels,as described above. These settings can impact the content that isdisplayed on the device. Further, these settings may impact which usersare able to control the device or otherwise interact with the device inone or more defined ways (e.g., audibly). For example, in family mode,only members of a particular group of users may be capable of invokingparticular operations on the device, in personal mode, only a singleuser may be capable of invoking those operations, and in meeting mode,any user may be capable of invoking the operations, as some examples.These operations could correspond to user agent operations, such asspeech commands, speech-based inferences, and the like.

Referring now to FIG. 4, a flow diagram is provided showing oneembodiment of a method 400 for selecting an interaction mode for adevice. At block 410, method 400 includes detecting a physicalconfiguration of a device. For example, configuration detector 120 candetect the physical configuration of device 202 shown in FIG. 2A basedon sensor data from configuration sensors 130. The physicalconfiguration can comprise a position of display 242A with respect todisplay 242B, where the position is physically adjustable. For example,a person could manipulate device 202 to change angle 250 therebyadjusting the position. Configuration detector 120 may detect thephysical configuration based on or in response to such a manipulation oradjustment to the position. In some cases, configuration detector 120determines or calculates the position from the sensor data (e.g., from asensor's physical measurements). The physical configuration could alsocomprise the orientation of the device with respect to support surface260 and this aspect of the physical configuration could be detected fromsensor data.

At block 420, method 400 includes selecting a configuration profilerepresenting the physical configuration. For example, configurationdetector 120 may select configuration profile 124A, which represents thephysical configuration. This can include mapping the sensor data to theconfiguration profile.

At block 430, method 400 includes activating an interaction modecorresponding to the configuration profile. For example, interactionmode selector 122 may automatically activate the interaction moderepresented by the configuration profile. This could include switchingfrom one interaction mode to another, and disabling/enabling variousmode I/O features to make available the mode I/O features of theinteraction mode.

At block 440, method 400 includes managing device interfaces using afeature of the interaction mode. For example, interface manager 128 canutilize available mode I/O features, such as to facilitate interactionwith one or more users of the device. This can include receiving and/orsending API or device calls to one or more components of the device inaccordance with the mode I/O features. For example interface manager 128may interface with software or services installed on the device. Thiscan include software 110 of FIG. 1. In some cases, interface manager 128is at least partially integrated into software 110 and/or one or moresoftware modules thereof.

Referring now to FIG. 5, a flow diagram is provided showing oneembodiment of a method 500 for selecting an interaction mode for adevice. At block 510, method 500 includes determining a physicalconfiguration of a device. For example, configuration detector 120 candetermine that device 202 is in a physical configuration correspondingto FIG. 2B. This could be determined in response to data from devicesensors, such as motion sensors. For example, a user could pick updevice 202 from the physical configuration of FIG. 2A, and devicesensors could provide readings that activate configuration detector 120,as an example. Subsequently, configuration detector 120 may determinedevice 202 is in physical configuration shown in FIG. 2B. In some cases,the position of the display(s) and/or regions thereof were initiallydetermined by configuration detector 120 for determining the physicalconfiguration of FIG. 2A. Based on an absence of sensor datacorresponding to the position (e.g., from an IMU), configurationdetector 120 may infer device 202 has remained in the same physicaldevice configuration. Thus, configuration detector 120 may refrain frompolling one or more sensors associated with detecting the physicaldevice configuration and/or need not process data from the sensors todetect the new physical configuration.

At block 520, method 500 includes selecting a configuration profilerepresenting the physical configuration. For example, configurationdetector 120 may select configuration profile 124B.

At block 530, method 500 includes switching interaction modes based onthe configuration profile. For example, interaction mode selector 122can switch or transition from an interaction mode corresponding toconfiguration profile 124A and the physical configuration of FIG. 2A tothe new interaction mode based on detecting the new physicalconfiguration of device 202. One or more mode I/O features may beswitched on or otherwise activated and/or switched off or otherwisedeactivated so as to ensure the mode I/O features of the new interactionmode are available to interface manager 128.

At block 540, method 500 includes managing device interface using theenabled interaction mode. For example, interface manager 128 can managedevice interfaces of device 202 using the mode I/O features of theactivated interaction mode.

With reference to FIG. 6, computing device 600 includes bus 610 thatdirectly or indirectly couples the following devices: memory 612, one ormore processors 614, one or more presentation components 616,input/output (I/O) ports 618, input/output components 620, andillustrative power supply 622. Bus 610 represents what may be one ormore busses (such as an address bus, data bus, or combination thereof).Although the various blocks of FIG. 6 are shown with lines for the sakeof clarity, in reality, delineating various components is not so clear,and metaphorically, the lines would more accurately be grey and fuzzy.For example, one may consider a presentation component such as a displaydevice to be an I/O component. Also, processors have memory. Theinventors recognize that such is the nature of the art and reiteratethat the diagram of FIG. 6 is merely illustrative of an exemplarycomputing device that can be used in connection with one or moreembodiments of the present invention. Distinction is not made betweensuch categories as “workstation,” “server,” “laptop,” “handheld device,”etc., as all are contemplated within the scope of FIG. 6 and referenceto “computing device.”

Computing device 600 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 600 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprise computerstorage media and communication media. Computer storage media includesboth volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer-readable instructions, data structures, program modules, orother data. Computer storage media includes but is not limited to, RAM,ROM, EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVDs) or other optical disk storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to store thedesired information and which can be accessed by computing device 600.Computer storage media does not comprise signals per se. Communicationmedia typically embodies computer-readable instructions, datastructures, program modules, or other data in a modulated data signalsuch as a carrier wave or other transport mechanism and includes anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia.

Memory 612 includes computer-storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, non-removable,or a combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 600includes one or more processors that read data from various entitiessuch as memory 612 or I/O components 620. Presentation component(s) 616present data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, etc.

I/O ports 618 allow computing device 600 to be logically coupled toother devices including I/O components 620, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc. The I/Ocomponents 620 may provide a natural user interface (NUI) that processesair gestures, voice, or other physiological inputs generated by a user.In some instances, inputs may be transmitted to an appropriate networkelement for further processing. An NUI may implement any combination ofspeech recognition, touch and stylus recognition, facial recognition,biometric recognition, gesture recognition both on screen and adjacentto the screen, air gestures, head and eye tracking, and touchrecognition associated with displays on the computing device 600. Thecomputing device 600 may be equipped with depth cameras, such asstereoscopic camera systems, infrared camera systems, RGB camerasystems, and combinations of these, for gesture detection andrecognition. Additionally, the computing device 600 may be equipped withaccelerometers or gyroscopes that enable detection of motion. The outputof the accelerometers or gyroscopes may be provided to the display ofthe computing device 600 to render immersive augmented reality orvirtual reality.

The present invention has been described in relation to particularembodiments, which are intended in all respects to be illustrativerather than restrictive. Alternative embodiments will become apparent tothose of ordinary skill in the art to which the present inventionpertains without departing from its scope.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the scopeof the claims below. Embodiments of the present invention have beendescribed with the intent to be illustrative rather than restrictive.Alternative embodiments will become apparent to readers of thisdisclosure after and because of reading it. Alternative means ofimplementing the aforementioned can be completed without departing fromthe scope of the claims below. Certain features and sub-combinations areof utility and may be employed without reference to other features andsub-combinations and are contemplated within the scope of the claims.

What is claimed is:
 1. A computer-implemented method comprising:detecting a physical configuration of a device based on sensor data fromone or more configuration sensors, the physical configuration comprisinga position of a first display region of the device with respect to asecond display region of the device, wherein the position is physicallyadjustable; selecting a configuration profile from a plurality ofconfiguration profiles based on the detected physical configuration ofthe device, each configuration profile being a representation of atleast one respective physical configuration of the device; activating aninteraction mode corresponding to the selected configuration profile,the interaction mode comprising a set of mode input/output (I/O)features available while the interaction mode is active; and managingdevice interfaces of the device using at least some mode I/O features inthe set of mode I/O features based on the activating of the interactionmode.
 2. The method of claim 1, wherein the first display region is of afirst display of the device and the second display region is of a seconddisplay of the device.
 3. The method of claim 1, wherein the detectingof the physical configuration comprises detecting a physical orientationof the device.
 4. The method of claim 1, wherein the first displayregion and the second display region are coupled together by a hingethat is configured to bend to adjust the position.
 5. The method ofclaim 1, wherein the first display region and the second display regionare each part of a bendable display of the device that is configured tobend to adjust the position.
 6. The method of claim 1, wherein theactivating of the interaction mode activates at least one mode I/Ofeature in the set of mode I/O features.
 7. The method of claim 1,further comprising after the managing of the device interfaces, inresponse to detecting the device is no longer in the physicalconfiguration, disabling at least one mode I/O feature in the set ofmode I/O features.
 8. The method of claim 1, wherein the activating theinteraction mode activates passive listening for speech input on thedevice.
 9. The method of claim 1, wherein the activating the interactionmode activates a personal assistant.
 10. The method of claim 1, whereinthe interaction mode is a far field interaction mode and the activatingof the interaction mode switches the device from a near fieldinteraction mode to the far field interaction mode.
 11. One or morecomputer storage media storing computer-useable instructions that, whenexecuted by one or more processors, cause the one or more processors toperform operations comprising: detecting a change of a device from afirst physical configuration to a second physical configuration based onsensor data from one or more configuration sensors, the second physicalconfiguration comprising a position of a first display region of thedevice with respect to a second display region of the device, whereinthe position is physically adjustable; selecting a configuration profilefrom a plurality of configuration profiles based on the detected secondphysical configuration of the device, each configuration profile being arepresentation of at least one respective physical configuration of thedevice; switching an active interaction mode from an interaction modecorresponding to the first physical configuration to an interaction modecorresponding to the second physical configuration based on the selectedconfiguration profile, each interaction mode comprising a set of modeinput/output (I/O) features available while the interaction mode isactive; managing device interfaces of the device using at least somemode I/O features in the set of mode I/O features based on theactivating of the interaction mode.
 12. The one or more computer storagemedia of claim 11, wherein the detecting of the change is based ondetecting an adjustment of an initial position of the devicecorresponding to the first physical position to the position of thesecond physical configuration.
 13. The one or more computer storagemedia of claim 11, wherein the switching of the active interaction modedeactivates each mode I/O feature in the interaction mode correspondingto the first physical configuration based on the mode I/O feature notbeing included in the interaction mode corresponding to the secondphysical configuration.
 14. The one or more computer storage media ofclaim 11, wherein the first physical configuration corresponds to afirst orientation of the device and the second physical configurationcorresponds to a second orientation of the device.
 15. The one or morecomputer storage media of claim 11, wherein the switching of the activeinteraction mode changes one or more privacy settings of a user agent.16. The one or more computer storage media of claim 11, wherein thesensor data comprises one or more physical measurements made by the oneor more configuration sensors.
 17. A system comprising: one or moreprocessors; and one or more computer storage media storingcomputer-useable instructions that, when executed by the one or moreprocessors, cause the one or more processors to perform operationscomprising: detecting a physical configuration of a device based onsensor data from one or more configuration sensors of the device, thedevice being bendable to adjust an angle between a first display regionof the device and a second display region of the device, the physicalconfiguration comprising the device being bent such that the angleallows the device to be supported by at least one edge of the device ona support surface, the at least one edge extending along the firstdisplay region and the second display region; selecting a configurationprofile from a plurality of configuration profiles based on the detectedphysical configuration of the device, each configuration profile being arepresentation of at least one respective physical configuration of thedevice; activating an interaction mode corresponding to the selectedconfiguration profile, the interaction mode comprising a set ofinput/output (I/O) mode features available while the interaction mode isactive; and managing device interfaces of the device using at least somemode I/O features in the set of mode I/O features based on theactivating of the interaction mode.
 18. The system of claim 17, whereinthe first display region is a first display on a first panel of thedevice and the second display region is a second display on a secondpanel of the device.
 19. The system of claim 17, wherein the devicecomprises a hinge coupling a first panel of the device to a second panelof the device, the hinge being adjustable to adjust the angle.
 20. Thesystem of claim 17, wherein the physical configuration comprises anorientation of the device with respect to the support surface.